■ n 
i -J 

; U 
: to 

I C 
{(A 



! H 

: o 



PATENT APPLICATION 
DOCKET NO. 1152-0009 



IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



Box PATENT APPLICATION 
Assistant Commissioner for Patents 
Washington, D.C. 20231 



"EXPRESS MAIL" Mailing Label No EJ434207393US . 

Date of Deposit November 19, 1999 



Sir: 

TRANSMITTAL LETTER 

Transmitted herewith for filing under 37 CFR § 1.53(b) is the patent application of: 

Inventor(s): Ramakrishna Pattikonda and Youling Lin 

For: SYSTEM AND METHOD FOR THREE-DIMENSIONAL 

SURFACE INSPECTION 

Enclosed are: 

[Y] Eighteen (1 8) pages of Patent Specification and Claim(s). 



X 1 Three (3) sheets of drawings. 



IT] A Combined Declaration and Power of Attorney (unsigned). 

Respectfully submitted, 




Steven W. Smith 
Reg. No. 36,684 



o 



u 




SMITH & DANAMRAJ, P.C. 
12900 Preston Road, Suite 1200, LB-15 
Dallas, Texas 75230-1328 
(972) 720-1202, ext. 229 



-1- 



PATENT APPLICATION 
DOCKET NO. 1 152-0009 



"EXPRESS MAIL" Mailing Label No EJ434207393US 

Date of Deposit November 19, 1999 



SYSTEM AND METHOD FOR 
THREE-DIMENSIONAL SURFACE INSPECTION 



BACKGROUND OF THE INVENTION 
Technical Field of the Invention 

The present invention relates to the inspection of structures on an 
object and, more specifically, to a system and method for optically 
10 obtaining three-dimensional information regarding the position of 
structures on the surface of an object. 

Descri ption of Related Art 

Optical inspection devices typically hold an object to be inspected 
15 under an overhead camera, and illuminate the object from a single or 
multiple light sources. Typically, the optical inspection device lights the 
object from several directions in order to fully illuminate the surface and 
any objects thereon. The overhead camera captures a two-dimensional 
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gray-scale (black-and-white) image of the object and structures. In a 
process commonly called convolution, this image is then sent to a 
computer which compares the image, pixel by pixel, to a stored image of 
an object with properly positioned structures thereon. If any differences 
between the captured image and the image of the object with properly 
positioned structures are detected, the computer has detected a defective 
object. A typical application of this inspection technology is in the 
inspection of printed circuit boards (PCBs). 

Other techniques for optical inspection such as the generation of 
structure grammar from captured images, tracing of structures to produce 
a set of primitives for the structure edges, the use of alignment techniques 
utilizing histograms to compensate for vibration and wobble of the support 
mechanism, and methods of automatic defect classification are disclosed 
in co-owned and co-pending U.S. patent application number 09/262,603 
entitled System and Method of Optically Inspecting Structures on an 
Object, and co-owned and co-pending U.S. patent application number 
09/338,880 entitled System and Method of Optically Inspecting Surface 
Structures on an Object, both of which are hereby incorporated by 
reference herein in their entireties. 

Existing optical inspection systems and methods are becoming very 
efficient at obtaining and analyzing surface structures and defects which 
reveal themselves in the two dimensions of the surface of the object. 
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However, there is still a problem in rapidly and efficiently obtaining 
detailed and accurate information about the height of the structures. Some 
prior art systems have used lasers to obtain height information. When a 
laser is pointed at a particular point on the surface of the object, a small 
dot is formed by the laser beam. If the angle of incidence of the laser 
beam is less than 90 degrees, and there is a structure at this point, the dot 
is displaced horizontally from the position where the dot would be if there 
was no structure. The taller the structure, the greater the displacement. 
The vertically mounted camera can then detect the displacement of the 
laser dot, and height information can be computed from the magnitude of 
the displacement. However, this is a slow and inefficient process which 
is not suitable for obtaining height information over the entire surface of 
the object. 

In order to overcome the disadvantage of existing solutions, it 
would be advantageous to have a system and method of rapidly and 
efficiently obtaining three-dimensional information regarding the position 
of structures on the surface of an object. The present invention provides 
such a system and method. 
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SUMMARY OF THE INVENTION 

In one aspect, the present invention is an optical inspection system 
for inspecting at least one structure on a surface of an object. The system 
includes a first visual light source which muminates the surface of the 

5 object and the structure with a light at a first visual frequency, and a first 
laser light source which illuminates the surface of the object with a narrow 
coherent laser beam simultaneously with illumination by the first visual 
light source. The laser beam is emitted at a second visual frequency that 
is different from the first visual frequency of the visual light source. The 

10 first laser light source is mounted off vertical on a movable mount which 
enables the laser beam to be directed over an area of interest on the 
surface of the object. The system also includes a color scan camera 
mounted vertically above the object. The camera has a first channel which 
captures an image of the muminated surface of the object and the structure 

15 at the first visual frequency, and a second channel which captures a path 
of the laser beam as it strikes the surface of the object and the structure at 
the second visual frequency. A computer then determines two- 
dimensional structure information from the image at the first visual 
frequency, and detennines height information for the structure from the 

20 path of the laser beam at the second visual frequency. The system may 
also include a second visual light source mounted on an opposite side of 
the object and illuminating the object at a third visual frequency. 
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Additionally, the system may include a second laser light source oriented 
90 degrees from the first laser light source. 

In another aspect, the present invention is a method of inspecting at 
least one structure on a surface of an object. The method includes the 

5 steps of illuminating the surface of the object and the structure with a first 
visual light at a first visual frequency, and simultaneously iUuminating the 
surface of the object with a first narrow coherent laser beam at a second 
visual frequency that is different from the first visual frequency. The first 
laser beam strikes the surface of the object at an angle of incidence less 

10 than 90 degrees. The laser beam is directed in a path covering an area of 
interest on the surface of the object. This is followed by capturing an 
image of the illuminated surface of the object and the structure at the first 
visual frequency utilizing a first channel of a color scan camera mounted 
vertically above the object. Simultaneously, the path of the laser beam at 

1 5 the second visual frequency is captured utilizing a second channel of the 
color scan camera as the laser beam strikes the surface of the object and 
the structure. Two-dimensional structure information is then determined 
from the image at the first visual frequency, and height information for the 
structure is determined from the path of the laser beam at the second visual 

20 frequency. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The invention will be better understood and its numerous objects 
and advantages will become more apparent to those skilled in the art by 
reference to the following drawings, in conjunction with the accompanying 
5 specification, in which: 

FIG. 1 is a simplified block diagram of a three-dimensional optical 
inspection system; 

FIG. 2 is an illustrative drawing of a component mounted on the 
surface of a printed circuit board (PCB) illustrating the height- 
10 measurement technique of the present invention; and 

FIG. 3 is a flow chart illustrating the steps of the method of the 
present invention. 

DETAILED DESCRIPTION OF EMBODIMENTS 

15 The present invention advantageously combines several new 

inspection techniques in order to rapidly and efficiently obtain three- 
dimensional information regarding the position of structures on the surface 
of an object. The exemplary embodiment described herein is discussed in 
the context of a PCB inspection system. However, the system and method 

20 described herein may also be utilized to inspect other types of structures 
and objects. 
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FIG. 1 is a simplified block diagram of a three-dimensional optical 
inspection system 10. The present invention utilizes laser light and multi- 
spectrum visual light together at the same time. A PCB 11 with 
components 12, or other formations such as solder blocks, on its surface 

5 is supported on a support mechanism 1 3 . Two multi-spectrum visual lights 
14 and 15 illuminate the PCB from opposite sides. Light 14 may be, for 
example green, while light 15 is blue. Two coherent red-light lasers 16 
and 17 are mounted off the vertical (i.e., the angle of incidence of then- 
laser beams is less than 90 degrees), and are mounted approximately 90 

10 degrees apart horizontally. Thus, if laser 16 points at the PCB from the 
front, laser 17 may point at the PCB from the right side or the left side. 
The lasers are mounted on movable mounts 1 8 and 19, enabling the lasers 
to be pointed at any point on the PCB surface. 

A color scan camera 20 is mounted above the PCB surface. The 

15 color scan camera performs a high speed scan of the PCB surface and 
utilizes red, green, and blue (RGB) channels to separate the signals of the 
laser light and the multi-spectrum visual lights. 

The inspection system is under the control of a control computer 2 1 . 
The control computer, or another dedicated computer, may also generate 

20 structure grammar from captured images, trace structures to produce sets 
of primitives for the structure edges, align multiple images utilizing 
histograms to compensate for vibration and wobble of the support 
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mechanism, perform height calculations, and perform automatic defect 
classification. 

In operation, the green light 14 and the blue light 15 are used to 
iUurninate the entire surface of the PCB. Simultaneously, the two coherent 

5 red-light lasers 16 and 17 are used to generate a series of parallel lines 
over the surface of the PCB 11. The lasers are mounted at 90 degree 
positions from each other and generate a grid of perpendicular red lines on 
the PCB surface. In addition, the lasers are strobed at a predetennined 
rate, so that both time-multiplexing and color-multiplexing can be used to 

10 separate the laser signal from the visual light signals generated by the 
green light and the blue light. The color scan camera utilizes its red, 
green, and blue channels to separate the red, green, and blue signals from 
the various light sources. 

The color scan camera 20 is controlled to point at the same position 

15 as one or both of the lasers. Then the camera and the laser are scanned 
together over the surface of the PCB and take height readings across the 
PCB. By using the laser grid lines, the present invention can obtain height 
information over the entire surface of the PCB in a single high-speed scan. 
When a laser grid line encounters an object on the surface of the PCB, 

20 there is a discontinuity in the laser grid line. The magnitude of the 
discontinuity is measured in order to determine the height of the object. 
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The color scan camera takes a continuous series of exposures as it 
scans. By controlling the exposure time, the resolution of the height 
information in the direction of the laser line is controlled. A very short 
exposure time provides a height reading for a desired point on the surface. 
A rapid series of short exposures provides a height profile over a region 
of the surface. For longer exposure times, a series of height measurements 
may be integrated to provide an average height over the entire surface or 
a region thereof. 

FIG. 2 is an illustrative drawing of a component 12 mounted on the 
surface of the PCB 1 1 illustrating the height-measurement technique of the 
present invention. Portions of two laser lines 31 and 32 are shown as they 
cross over the component 12. Dotted lines 33 and 34 illustrate the 
imaginary positions that lines 31 and 32 would present to the camera 20 
if the structure was not present. Lines 3 1 and 32 are displaced from dotted 
lines 33 and 34 by distance "d". The displacement distance is dependent 
on the height of the structure and the angle by which the laser is mounted 
off the vertical. When the laser is mounted farther off the vertical, smaller 
height differences cause a greater displacement. Therefore, smaller height 
differences can be detected. 

FIG. 3 is a flow chart illustrating the steps of the method of the 
present invention. At step 41, the PCB 1 1 to be inspected is positioned on 
the support mechanism 13. At step 42, the PCB is illuminated with blue 
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visual light, green visual light, and red laser light. At step 43, the color 
scan camera 20 is aligned with the position of one of the lasers 16 and 17. 
The color scan camera and the laser are then scanned together across the 
surface of the PCB at 44. The color scan camera uses its red, green, and 

5 blue channels to separate the signals from the three light sources. The red 
signal may be further differentiated, as noted above, by strobing the laser 
and using time-multiplexing to extract the signal. At 45, the signals from 
the green and blue channels are analyzed using known techniques to obtain 
two-dimensional information regarding the structure of the component 

10 mounted on the PCB. At 46, the signal from the red channel is analyzed 
using the techniques illustrated in FIG. 2 to obtain height information for 
the component. As noted above, this information may be utilized to 
determine a height profile, or the information may be integrated over a 
period of time at step 47 to obtain an average height value. 

15 By utilizing the method of the present invention, the present 

invention can obtain three-dimensional structure information over the 
entire surface of the PCB in a single high-speed scan. 

It is thus believed that the operation and construction of the present 
invention will be apparent from the foregoing description. While the 

20 system and method shown and described has been characterized as being 
preferred, it will be readily apparent that various changes and 
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modifications could be made therein without departing from the scope of 
the invention as defined in the following claims. 
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WHAT IS CLAIMED IS: 



1 1. An optical inspection system for inspecting at least one 

2 structure on a surface of an object, said system comprising: 

3 a first visual light source which illuminates the surface of the object 

4 and the structure with a light at a first visual frequency; 

5 a first laser light source which Hluminates the surface of the object 

6 with a narrow coherent laser beam simultaneously with illumination by the 

7 first visual light source, said laser beam being at a second visual frequency 

8 that is different from the first visual frequency of the visual light source, 

9 said first laser light source being mounted off vertical on a movable mount 

1 0 which enables the laser beam to be directed over an area of interest on the 

1 1 surface of the object; 

12 a color scan camera mounted vertically above the object, said 

13 camera having a first channel which captures an image of the illuminated 

14 surface of the object and the structure at the first visual frequency, and a 

15 second channel which captures a path of the laser beam as it strikes the 

16 surface of the object and the structure at the second visual frequency; and 

1 7 a computer which determines two-dimensional structure information 

18 from tibie image at the first visual frequency, and determines height 

19 information for the structure from the path of the laser beam at the second 

20 visual frequency. 
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1 2. The optical inspection system of claim 1 further comprising 

2 a second visual light source mounted on an opposite side of the object 

3 from the first visual light source, said second visual light source 

4 illuminating the surface of the object and the structure with a light at a 

5 third visual frequency. 

1 3. The optical inspection system of claim 2 wherein the color 

2 scan camera includes a third channel which captures an image of the 

3 illuminated surface of the object and the structure at the third visual 

4 frequency. 

1 4. The optical inspection system of claim 3 wherein the 

2 computer also determines two-dimensional structure information from the 

3 image at the third visual frequency. 

1 5. The optical inspection system of claim 4 wherein the color 

2 scan camera includes means for making a continuous series of exposures 

3 as the camera scans the surface of the object. 
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1 6. The optical inspection system of claim 5 wherein the means 

2 in the color scan camera for making a continuous series of exposures 

3 includes means for varying the length of each exposure. 

1 7. The optical inspection system of claim 6 wherein the 

2 computer includes means for integrating the height information over the 

3 length of an exposure to calculate an average height. 

1 8. The optical inspection system of claim 1 further comprising 

2 a second laser light source mounted on a side of the object which is 

3 displaced 90 degrees from the first laser light source, said second laser 

4 light source illuminating the surface of the object in a path that is 

5 perpendicular to the path ihuminated by the first laser light source. 

1 9. A method of inspecting at least one structure on a surface of 

2 an object, said method comprising the steps of: 

3 illuminating the surface of the object and the structure with a first 

4 visual light at a first visual frequency; 

5 simultaneously illuminating the surface of the object with a first 

6 narrow coherent laser beam at a second visual frequency that is different 

7 from the first visual frequency, said first laser beam striking the surface of 

8 the object at an angle of incidence less than 90 degrees; 
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9 directing the laser beam in a path covering an area of interest on the 

1 0 surface of the obj ect; 

1 1 capturing an image of the illuminated surface of the object and the 

12 structure at the first visual frequency utilizing a first channel of a color 

13 scan camera mounted vertically above the object; 

14 simultaneously capturing the path of the laser beam at the second 

1 5 visual frequency utilizing a second channel of the color scan camera as the 

16 laser beam strikes the surface of the object and the structure; 

1 7 determining two-dimensional structure information from the image 

18 at the first visual frequency; and 

1 9 determining height information for the structure from the path of the 

20 laser beam at the second visual frequency. 



1 10. The method of inspecting at least one structure on a surface 

2 of an object of claim 9 further comprising illuminating the surface of the 

3 object and the structure with a second visual light at a third visual 

4 frequency, the second visual light being mounted on an opposite side of 

5 the object from the first visual light. 
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1 11. The method of inspecting at least one structure on a surface 

2 of an object of claim 10 further comprising simultaneously capturing an 

3 image of the illuminated surface of the object and the structure at the third 

4 visual frequency utilizing a third channel of the color scan camera. 

1 12. The method of inspecting at least one structure on a surface 

2 of an object of claim 1 1 further comprising determining two-dimensional 

3 structure information from the image at the third visual frequency. 

1 13 . The method of inspecting at least one structure on a surface 

2 of an object of claim 12 wherein the step of simultaneously capturing the 

3 path of the laser beam includes making a continuous series of exposures 

4 with the color scan camera as the camera scans the surface of the object. 



1 14. The method of inspecting at least one structure on a surface 

2 of an object of claim 1 3 wherein the step of making a continuous series of 

3 exposures includes varying the length of each exposure. 

1 15. The method of inspecting at least one structure on a surface 

2 of an object of claim 14 further comprising integrating the height 

3 information over the length of an exposure to calculate an average height. 
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1 16. The method of inspecting at least one structure on a surface 

2 of an object of claim 9 further comprising iUuminating the surface of the 

3 object with a second laser light source mounted on a side of the object 

4 which is displaced 90 degrees from the first laser light source, said second 

5 laser light source iUuminating the surface in a path that is perpendicular to 

6 the path illuminated by the first laser light source. 
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ABSTRACT OF THE DISCLOSURE 

An optical inspection system and method for inspecting a 
component on a printed circuit board (PCB) which determines three- 
dimensional information in a single scan. A first visual light source 

5 illuminates the PCB surface and component with a green light while a 
second visual light source iUuminates the PCB and component with a blue 
light. At least one laser light source simultaneously illuminates the surface 
of the PCB with a narrow coherent red-light laser beam. The laser light 
source is mounted off vertical on a movable mount which enables the laser 

10 beam to be directed over an area of interest on the surface of the PCB. 
The system also includes a color scan camera mounted vertically above the 
PCB. The camera has red, green, and blue channels. The green and blue 
channels capture an image of the illuminated surface of the PCB which is 
used by a computer to determine two-dimensional information about the 

1 5 component. The red channel captures a path of the laser beam as it strikes 
the surface of the PCB and the component. The computer uses the path 
to determine height information for the component. 
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COMBINED DECLARATION AND POWER OF ATTORNEY 



As a below named inventor, I hereby declare that: 

My residence, post office address, and citizenship are as stated below next to my name; and 

I verily believe that I am an original, first, and joint inventor of the subject matter which is 
claimed and for which a patent is sought on the invention entitled: 



SYSTEM AND METHOD FOR THREE-DIMENSIONAL SURFACE INSPECTION 



the specification of which: 

is attached hereto. 

— was filed on as Application Serial No. and was 

amended on (if applicable). 

I hereby state that I have reviewed and understand the contents of the above identified 
specification, including the claims, as amended by any amendment referred to above. 

I acknowledge the duty to disclose to the Office all information known to me to be material 
to the patentability of this application as defined in 37 CFR § 1.56. 

I hereby claim foreign priority benefits under 35 U.S.C. § 1 19 of any foreign application(s) 
for patent or inventor's certificate listed below and have also identified below any foreign application 
for patent or inventor's certificate having a filing date before that of any application on which priority 
is claimed: 



Country 



Number 



Date Filed Priority 
Claimed 
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I hereby claim the benefit under 35 U.S.C § 120/365/1 19 and 37 CFR § 1.78 of any United 
States application(s) listed below and, insofar as the subject matter of each of the claims of this 
application is not disclosed in the prior United States application in the manner provided by the first 
paragraph of 35 U.S.C. § 1 12, 1 acknowledge the duty to disclose material information as defined 
in 37 CFR §1.56 which occurred between the filing date of the prior application and the national 
or PCT international filing date of this application: 

Application Serial No. Filing Date Status (patented, pending) 



I hereby appoint the following attorneys and/or agents to prosecute this application and to 
transact all business in the Patent and Trademark Office connected therewith: STEVEN W. 
SMITH, Reg. No. 36,684 and SHREEN K. DANAMRAJ, Reg. No. 41,696 of the firm of Smith & 
Danamraj, P.C., 12900 Preston Road, Suite 1200, LB-15, Dallas, Texas 75230-1328. 

Address all telephone calls and correspondence to: 

Steven W. Smith 

SMITH & DANAMRAJ, P.C. 

12900 Preston Road, Suite 1200, LB-15 

Dallas, Texas 75230-1328 

(972) 720-1202, ext. 229 

I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these statements 
were made with the knowledge that willful false statements and the like so made are punishable by 
fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that 
such willful false statements may jeopardize the validity of the application or any patent issued 
thereon. 
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